{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "9438aa74",
   "metadata": {},
   "source": [
    "## Storing data on disk in CSV format"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "6da5a540",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from openbb import obb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "0be2de01",
   "metadata": {},
   "outputs": [],
   "source": [
    "obb.user.preferences.output_type = \"dataframe\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "63b87610",
   "metadata": {
    "lines_to_next_cell": 2
   },
   "source": [
    "Function to fetch historical stock data for a given symbol and date range, and add a 'symbol' column"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "798975af",
   "metadata": {
    "lines_to_next_cell": 2
   },
   "outputs": [],
   "source": [
    "def get_stock_data(symbol, start_date=None, end_date=None):\n",
    "    data = obb.equity.price.historical(\n",
    "        symbol,\n",
    "        start_date=start_date,\n",
    "        end_date=end_date,\n",
    "        provider=\"yfinance\",\n",
    "    )\n",
    "    data.reset_index(inplace=True)\n",
    "    data[\"symbol\"] = symbol\n",
    "    return data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "50da6bdb",
   "metadata": {
    "lines_to_next_cell": 2
   },
   "source": [
    "Function to save the fetched stock data to a gzipped CSV file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "aef69e67",
   "metadata": {
    "lines_to_next_cell": 2
   },
   "outputs": [],
   "source": [
    "def save_data_range(symbol, start_date=None, end_date=None):\n",
    "    data = get_stock_data(symbol, start_date, end_date)\n",
    "    data.to_csv(f\"{symbol}.gz\", compression=\"gzip\", index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27a04bc5",
   "metadata": {
    "lines_to_next_cell": 2
   },
   "source": [
    "Function to read stock data from a gzipped CSV file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "a977e57c",
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_data(symbol):\n",
    "    return pd.read_csv(\n",
    "        f\"{symbol}.gz\",\n",
    "        compression=\"gzip\",\n",
    "        index_col=\"date\",\n",
    "        usecols=[\"date\", \"open\", \"high\", \"low\", \"close\", \"volume\", \"symbol\"],\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "81884720",
   "metadata": {},
   "source": [
    "Save data for the stock symbol \"PLTR\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "4df17ff7",
   "metadata": {},
   "outputs": [],
   "source": [
    "save_data_range(\"PLTR\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dc4a0d1c",
   "metadata": {},
   "source": [
    "Get the saved data for \"PLTR\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "d283ec54",
   "metadata": {},
   "outputs": [],
   "source": [
    "pltr = get_data(\"PLTR\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4959baa9",
   "metadata": {},
   "source": [
    "Save the DataFrame 'df' to a CSV file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "483a13d7",
   "metadata": {},
   "outputs": [],
   "source": [
    "pltr.to_csv(\"market_data.csv\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3e53a87",
   "metadata": {},
   "source": [
    "Save the DataFrame 'df' to a tab-separated file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "680d5918",
   "metadata": {},
   "outputs": [],
   "source": [
    "pltr.to_csv(\"market_data.tsv\", sep=\"\\t\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e65b9ce",
   "metadata": {},
   "source": [
    "Save specific columns of 'df' to a CSV file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "48e59bb6",
   "metadata": {},
   "outputs": [],
   "source": [
    "pltr.to_csv(\"market_data.csv\", columns=[\"open\", \"close\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8df91bec",
   "metadata": {},
   "source": [
    "Save 'df' to a CSV file with a specific date format"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "07962aa6",
   "metadata": {},
   "outputs": [],
   "source": [
    "pltr.to_csv(\"market_data.csv\", date_format=\"%Y-%m-%d\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "46b7b7e7",
   "metadata": {},
   "source": [
    "Read the first 10 rows of the CSV file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "657cfaca",
   "metadata": {},
   "outputs": [],
   "source": [
    "pltr = pd.read_csv(\"market_data.csv\", nrows=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b55bc115",
   "metadata": {},
   "source": [
    "Skip the first 10 rows and read the rest of the CSV file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "71488696",
   "metadata": {},
   "outputs": [],
   "source": [
    "pltr = pd.read_csv(\"market_data.csv\", skiprows=range(1, 11))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3590bcae",
   "metadata": {},
   "source": [
    "Read the CSV file and specify 'NULL' as the value to be treated as NaN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "ab73ea34",
   "metadata": {},
   "outputs": [],
   "source": [
    "pltr = pd.read_csv(\"market_data.csv\", na_values=\"NULL\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f9e5fd59",
   "metadata": {},
   "source": [
    "**Jason Strimpel** is the founder of <a href='https://pyquantnews.com/'>PyQuant News</a> and co-founder of <a href='https://www.tradeblotter.io/'>Trade Blotter</a>. His career in algorithmic trading spans 20+ years. He previously traded for a Chicago-based hedge fund, was a risk manager at JPMorgan, and managed production risk technology for an energy derivatives trading firm in London. In Singapore, he served as APAC CIO for an agricultural trading firm and built the data science team for a global metals trading firm. Jason holds degrees in Finance and Economics and a Master's in Quantitative Finance from the Illinois Institute of Technology. His career spans America, Europe, and Asia. He shares his expertise through the <a href='https://pyquantnews.com/subscribe-to-the-pyquant-newsletter/'>PyQuant Newsletter</a>, social media, and has taught over 1,000+ algorithmic trading with Python in his popular course **<a href='https://gettingstartedwithpythonforquantfinance.com/'>Getting Started With Python for Quant Finance</a>**. All code is for educational purposes only. Nothing provided here is financial advise. Use at your own risk."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "43d4b6cb-1c99-4cba-a041-59c78d54d007",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "jupytext": {
   "cell_metadata_filter": "-all",
   "main_language": "python",
   "notebook_metadata_filter": "-all"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
